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Abstract. In this paper, the sharp fc-order Sobolev inequalities in the hyperbolic space H™ are 
established for all k = 1,2, This solves a long-standing open problem posed by Aubin [A4, 
p.176-177] for W k > 2 (W n ) 



§1. Introduction 

It is well-known that sharp Sobolev inequalities are important in the study of partial differ- 
ential equations, especially in the study of those arising from geometry and physics. A striking 
example where they have played an essential role in the Riemannian context is given by the 
solution of the famous Yamabe problem (see [Ai] and [S]). 

The classical Sobolev inequality (also called Sobolev imbedding theorem, see [So]) says that 
given an integer k G N, if 2k < n, then there is a constant C(n,k), depending only upon n 
and k, such that for all u G W k ' 2 (R n ) 1 

(1.1) ||'u|| I/ 2»/(»-2fe)(] R ») < C(n,k)\\u\\ W k,2Qg n ^, 

or equivalently, W k,2 (M. n ) is continuously embedded in L 2n /( n-2A 0(R n ). 

For the case k = 1, Talenti [T] and Aubin [A2, A3] (see also [R], [Ro]) proved that for any 
u G W 1,2 (W l ), 

/ r \ (n-2)/n „ 

(1.2) / \u(x)\ 2n/{n - 2) dx) <Ai / \Vu\ 2 dx, 
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where the best Sobolev constant is 

(1.3) Ax = 



( o^ 2/n' 

n(n — 2)uJn 



and u n is the surface area of the sphere § n = {x G W n+1 \\x\ = \J x\ + • • • + x n+i = -*-}• The 
equality sign holds in (1.2) if and only if u has the form: 



u(x) = a 



T 2 + \x — X \ 2 

27 



where a G R 1 , < r G I 1 and x G R n . 

By solving the Yamabe problem, Aubin proved (see [Ai] and [A2]) that the above sharp 
inequality is equivalent to the sharp Sobolev inequality on the sphere § n , written as 

/ r \ (n-2)/n « . 

(1.4) / \u{y)\ 2n '^dy) <Aj \ Vu\ 2 dy + u~ 2 / n \u(y)\ 2 dy 
for any u G W 1,2 (S n ). The equality sign holds if and only if u has the form: 

(^2 _ 1 \(n-2)/4 

u(y)= (^-oosr)(n-2)/2 » ^> X ' 

where r is the distance from y to yo on S n , and yo is a fixed point on S n . Note that the 
inequality (1.4) is sharp because the two best constants Ai and Un 2 ^ n cannot be lowered (see 
[Hi] or [H 3 ]). ' 

There has been much work on such inequalities (for k = 1) and their applications. See, for 
example, Trudinger [Tr], Moser [M], Aubin [A 2 , A 3 ], Talenti [T], Bliss [Bl], Lieb [Li, L 2 ], Brezis 
and Nirenberg [BN], Cherrier [C], Brezis and Lieb [BL], Carleson and Chang [CC], Escobar 
[E], Carlen and Loss [CL], Beckner [B], Adimurthi and Yadava [AY], Hebey and Vaugon [HVi, 
HV 2 ], Hebey [Hi, H 2 ], Li and Zhu [LZi, LZ 2 ], Zhu [Z], Aubin, Druet and Hebey [ADH], Aubin 
and Li [AL], and the references therein. 

For order k = 2, the sharp Sobolev inequality in R n was studied by Edmunds, Fortunato 
and Jannelli [EFJ] (also see Lions [Li]) and was explicitly given in [DHL]. On the sphere S n , 
the second-order sharp Sobolev inequality has also been established by Djadli, Hebey and 
Ledoux [DHL]. 

For higher order k, Cotsiolis and Tavoularies [CT] (also see [L]) obtained the sharp Sobolev 
inequality (via Lieb's sharp Hardy-Littlewood- Sobolev inequality): for any u G VF fc ' 2 (R n ), 

n — 2k 

(1.5) ([ \u(x)\^dx] " <A k I \A k / 2 u\ 2 dx, 
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where 

\A k / 2 u\ 2 = 

and the best Sobolev constant is 



|A fc / 2 w| 2 when k is even, 
| V ( A (fc-i)/2 u )|2 when k ig odd5 



(1.6) 



Afc = 



o2fc , ,-( 2k )/n 



n[n- 2k] [n 2 - (2(k - 1)) ] [n 2 - (2(k - 2)) ] ■ ■ ■ [n 2 - 2 2 ] 
The equality sign holds in (1.5) if and only if u has the form: 



(1.7) 



w(x) = a 



t + \x — Xq\ 
2~7 



k— — 



where a G R 1 , < r £ I 1 and x G R n . 

In [L], the author proved the following total optimal Sobolev inequalities on the sphere § n : 
Let (§ n , g) be the standard unit sphere of R n+1 , n > 2k, and let q = (2n)/(n — 2k). Assume 
that 



fe-i 



m=0 



is the k-th standard operator on S n whose coefficients A km is given by Qk = Qi(Qi 
2) • • • (Qi - k(k - 1)) with Qi = A g + Then, for any u G 17 fc > 2 (S n ), 



(1.8) 



lu|W„ 



2/9 



I (\A k ^u\ 2 +J2A km \A^uAdV g 
Jsn V m=0 / 



where A fc is the best Sobolev constant in R n . The equality sign holds if and only if u has the 
form: 

(/P-l) 2 ^ 



«/3(r) = 



and 



(/3 — cos r) ™ 2 



I 



2fe J 



/3>1, 



Q k {u-^u p ) = —{u-^upy- 1 on S», 



where r is the distance from y to yo on S n , and yo is a fixed point on S n . The best constants 
AkA m k in (1.8) are total optimal. 

In this paper, for any order k = 1, 2, • • • , we shall establish the sharp /c-order Sobolev in- 
equality in the hyperbolic space H n of constant sectional curvature —1. Our best Sobolev 
constants are also total optimal. This solves a long-standing open problem posed by Aubin 
[A 4 , p. 176-177] for W k ' 2 {W l ). In addition, we give an explicit representation (by an impor- 
tant recursive formula) for the high-order Paneitz operators P k in H n (P k are the conformal 
analogues of the power Laplacians A fc on the flat R n ). 
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We point out that the method here is more difficult than and quite different from that of 
[L] because there does not exist the extremal function for the sharp Sobolev inequality in HP. 
Fortunately, we can still get total optimal Sobolev constants in HP. 



§2. High-order Paneitz operators in the hyperbolic space HI 



The hyperbolic n-space HP (n > 2) is a complete simple connected Riemannian manifold 
having constant sectional curvature equal to —1, and for a given dimensional number, any two 
such spaces are isometric ([W]). There are several models for HP, the most important being 
the half-space model, the ball model, and the hyperboloid or Lorentz model, with the ball 
model being especially useful for questions involving rotational symmetry. We will only use 
the ball model in this paper. 

Let B n = {x = (xx, ■ ■ ■ ,x n ) e R n \(xl H h a^) 1/2 < 1} be the unit ball in the Euclidean 

space IP. For B n , if we endow with the Riemann metric 



4\dx\ 


2 


(1- 




X 


2) 2 



then the sectional curvature becomes the constant —1. Furthermore, if we now define spherical 
coordinates about x = by 

(2.1) x = t£, £ = tanh(r/2), 

where t e [0, 1), r G [0, oo), C G § n_1 , then we obtain the metric 

ds 2 = (dr) 2 + (sinh 2 r)|dC| 2 - 

Note that for each iGB n , t and r are the Euclidean and the hyperbolic distances from to x, 
respectively. One easily sees that, in the ball model, the geodesies emanating from the origin 
are given by straight lines emanating from the origin, and their length to the boundary S n_1 
is infinite. 

Let A h be the Laplacian on M n with the metric h, and let F : HP -> R 1 e C 2 , with 



F(y(r,()) = f(r,(). 
We have by direct calculation (see [Chi, p. 40]) that 

(A h F)(y(rX)) = -(sinhr) 1 "^ ^(sinhr)- 1 ^^ + (sinhr)- 2 £ c /, 

where, when writing C^f, we mean that /|S(r) is to be considered as a function on S n_1 with 
associated Laplacian C. 
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If / is a radial function on (HP, h) (i.e., function that depends only on distance from 
on HP), then the corresponding Laplacian takes the following simple form (see also [Ch 2 , 
p.180-181]): 



(2.2) 



A /l /(r,C) = -(sinhr) 1 -|- 



(sinhrp- 1 ^- 
or 



We denote such a simple form as A r . Similarly, for any positive integer m we can define 
the m th -iterated operator AP on the set of radial functions as the following: For any radial 
function / £ C 2m (HP), 



AP/(r) = -(sinhr 



,1-n 



d_ 

dr 



(sinr)' 



dr 



m = 1,2, 



Our first task is to determine the best Sobolev constants by the Euler-Lagrange equation 
(2.10) and a known radial function on HP. 

From the introduction, by setting r = (j^) 1 ^ 2 in (1.7) we know that for any integer k, 



(2.3) 



i-^y/vi + ^fW 2 



l + P 



k ^ 



(o < p < 1) 



are the extremal functions for the sharp /c-order Sobolev inequality (1.5) in Euclidean space 
M n , and 



(2.4) 



in 



where q = n 2 J! 2k ■ However, there is not extremal function for the sharp /c-order Sobolev 
inequality in the unit ball B n C M. n . In fact, the following Lemma holds 

Lemma 2.1. Let Q be a bounded open subset of W 1 , n > 2k, and let q = n 2 ^ k ■ Assume 
that Sjt is a constant defined by 



inf 



In 


A k/2 U 


2 dx 


(L 


\u 


idx) 2 /i 



Then = Moreover, the infimum is not attained. 

Proof. Standard rescaling arguments show that the infimum in the definition of does not 
depend on O. Since 

1 . . / R „ \A k / 2 u\ 2 dx 



inf 

Afc «6W fe . 2 (t»)\{0} 



(J Rn \u\idx) 



2/?' 



it immediately follows that < J^. 
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On the other hand, > For suppose < Then, there is a sequence {uj} in 
C °°(R n )\{0} such that " 



lim 

(/ R -I«il fl ^) 



2/9 



Thus for large enough j, y* 2 J 2/q < and this inequality contradicts the definition 

{f R n Wj\ 9 dx) ~ fe 
of Sfc and its independence of O. We conclude that H*. = 

Finally we show that the infimum in the definition of J- is not attained. Suppose that there 

exists it G W fc ' 2 (n) \ {0} such that j n ^ A — "L^ = J-. The extension of ti by zero outside 

[In \ u \ qdx ) " h 

O leads to a minimum of ^ A — ^ , but the minimizing function is not of the type (1.7). 

(/ R „|u|«d«) 

This is a contradiction, which proves the Lemma. □ 
It is easy to verify that 

L \A k / 2 G \ 2 dx L \A k / 2 u\ 2 dx i 
(2.5) lim " — 2fc = inf ^ = __. 

Let cr be the conformal map from the Euclidean ball B n to the hyperbolic space HP defined 
by (2.1). Then the Jacobian of a is J a = dJ^p ) • Suppose / is a smooth function defined 
in the Euclidean ball B n . Lift it to the hyperbolic space HP 1 by formula: 

(J a ) 1 ^F(y) = f(x), xGB n , ye IF, 



2 h-n. 

(2.6) F (y) = (— p^)* 2/(x ), xe^yeW 1 . 

By this method, we can transform functions from the Euclidean ball B n to HP 1 . 

It is not difficult to check that the above function Gp{x) (defined in the Euclidean ball 
B n ) is lifted to the function (1 - /? 2 )( n_2fc )/ 4 (coshr - (3) k ~% defined in HP. In fact, since 
|x| = tanh |, we have 

1 + tanh 2 § 1 tanh 2 £ 



1 + 1 


x\ 


2 


1 - 


X 


2 



+ 



1 — tanh | sech | sech | 

2 r n r 2 r 

— cosh — h sinh - = 2 cosh 1 = cosh r. 

2 2 2 



It follows that 



1 4- i±£|-rl 2 i /1 i I 12 \ i 

i + T^FI 1 j 1 + \x\ n \ ! 



(") - T ^ = r^(i^-/') = r^(^'-/')> o</><i- 
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Thus 



_\ K 2 



1-/3\ 17 V1 + T 



1+^1^12 
-(3 l x l 



1-/3\ 1/2 1 + T^N 2 



1 



1 + 

(1 -/3 2 )"i - (1 



2 

1 + T 



k 



1+/3| T |2 
/3 l x l 



.r 



(2.8) 



(1 -/3 2 )-2(coshr-/3) 



7. _n_ 

2 



(l_^(n-2fc)/4( coshr _^fc-5 



Now, we calculate the coefficients of the following Euler-Lagrange equation (2.10) by the 
radial function (1 - /3 2 )( n - 2fc )/ 4 (cosh r-(3) k ~^ (0 < < 1). 

Lemma 2.2. Assume that k is a positive integer, and assume that (HP, h) is the hyperbolic 
space, n > 2k. Given (3 G (0, 1). Then by the radial function 

(2.9) Vfc(r) = (l-/3 2 ) (n - 2fc)/4 (coshr-/3) fc -^, < r < +00, 

and by the following differential equation 



k-1 



(2.10) 



A h r Mr) + akm&?il>k{r) = b k ^ k (r) 



^ + 2fc 



m=0 



we can uniquely determine the constants b k and a k o, • • • > &k,k-i- In particular, 

n(n - 2) 

rr - 2n - \ . n(n - 4)(n 2 - 2 2 ) 



«io = h = 



021 



■ a 2 o = b 2 = 



2 ' ~ m 2 4 

3n 2 - 6n - 32 3n 4 - 12n 3 - 52n 2 + 128n + 192 
«32 — : , «31 — 



16 



- «30 = h 



n(n-6)(n 2 -4 2 )(n 2 -2 2 ) 
26 



3 3 27 
a 43 = -(n 2 -2n- 20), a 42 = ^n 4 - -n 3 - — n 2 + 30n + 108, 

8 2 2 



n 6 3 29 

a 4 i = -( n 5 - 3n 4 + — n s + 39n 2 - 108n - 144), 

v 16 8 2 ' 



a 4 o — 64 



n(n - 8)(n 2 - 6 2 )(n 2 - 4 2 )(n 2 - 2 2 ) 



2 8 



5n 2 - lOra - 160 



a 5 4 — 



a 50 — ^5 — 



n(n - 10)(n 2 - 8 2 )(n 2 - 6 2 )(n 2 - 4 2 )(n 2 - 2 2 ) 

2io 
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and 



(2.11) 



(-l) fc a fc0 =b k 

n[n-2k] \n 2 - (2(k - l)f 



TV 



-(2(k-2)) 2 ...[n 2 -2 2 ] 



2 2k 



for k = 1,2, • - 

Proof. We shall calculate the corresponding coefficients in (2.10) only for order k = 1,2,3. 
Other orders can similarly be done. 

(i) For k = 1, since ipi(r) = (1 - /3 2 )( n ~ 2 )/ 4 (coshr - (3) 1 ~^ we have 



A r ^i = — (sinhr) 



l-n 



d_ 

dr 



(sinhr) 



n-1^1 



- (1 -/^("-^(sinhr) 1 " 



dr 
d 



dr 



(sinhr) n • (1 — — )(coshr — j3) 2 sinhr 
d 



= - (1 - /3 2 ) (n " 2 )/ 4 (l - ^(sinhr) 1 ""^ [(coshr - (3)~% sinh n r] 



--(1 - /3 2 )( n - 2 )/ 4 (l - ^) (^)(coshr - P)-*- 1 sinh 2 r - n(coshr - coshr 



By setting 
we get 



Tl + 2 

n-2 



A r Vi + aio^i = hi^i , 



(1 — — ) ( — )(coshr — P) 2 1 sinh 2 r — n(coshr — p) ? coshr 



+ ai (coshr - P) 1 '^ = 6i(l - /3 2 )(coshr - /J) - ?" 1 , 



— — l 



i.e., 



(1 ) ( — ) sinh 2 r — n( coshr — p) coshr 

+ ai (coshr - P) 2 = &i(l - P 2 ). 



Thus 



-2 (1 "2 )+ai ° 



cosh r + 



+ 



1 - - J /3 - 2a 10 /3 



coshr 



= 0. 



This leads to the following system of equations with unknown constants aio and b\: 

-f(l-f)+a 10 = 0, 
/3 [n(l - § ) - 2a 10 ] = 0, 
-f(l-f)+a 10 /3 2 -6 1 (l-/3 2 )=0 
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which has the unique solution 



n(n — 2) n(n — 2) 
«io = : , Oi = : 



(ii) For k = 2, by if> 2 (r) = (1 - /3 2 )( n - 4 )/ 4 (coshr - (3) 2 ~^ we have 



A^ 2 = -(sinhr) 1 -|- 



(sinhr) 



n-1^2 



dr 



- (1 — (sinhr) J 



<9r 



(sinhr) n_1 (2 )(coshr — sinh', 



= - (1 - /? 2 ) ™4 4 (2 - ^) (1 - ^) (coshr - /?) 2 sinh 2 r + n(coshr - (5) 1 2 coshr] 
Furthermore, 



A 2 ^ 2 = - (sinhr) 1 ~ n ^- 

or 



(sinhr) 



n _ 1 a(A r -0 2 ) 

<9r 



= (1 - /3 2 )^ (2 - ^(sinhr) 1 "" ^ [(1 - ^)(-^)(coshr - P)-^- 1 (smhr) n+2 
+ (n + 2) (l — — ) (coshr — [3)~% (sinhr) n coshr + n(coshr — Z?) 1- ^ (sinhr)' 

Z 

= (1 - /3 2 )^ (2 - [(1 - £)(-£)(-£ - l)(coshr - /3)"t- 2 s inh 4 r 
+ 2(n + 2)(l- -)(--)(coshr -/3)""2 i " 1 (sinhr) 2 coshr 
+ re(n+2)(l- ^) (cosh r - cosh 2 r 

Z 



By putting 



we get that 



+ 2(n + 1)(1 - -)(coshr - (3)s sinti r 

Z 

+n 2 (coshr — /3) 1- ^ coshr] . 



A 2 V>2 + 0,21 A r ^2 + a2O"02 = foV 7 : 



71 + 4 

n — 4 



(2 



n. 



-2(n + 1)(1 - -)(coshr - /3)"^ + (1 - -)(n 2 + 4n + 2)(coshr - (3)~% cosh 2 r 
2 2 



+n 2 (cosh r — /?) 1 2 coshr + 2(n + 2)(1 — ^)(— ^)(coshr — j3) 2 1 (cosh 3 r — coshr) 

o Z 

+(1 " ?)(-?)(-? " l)(cosh 2 r - /3)-^- 2 (cosh 4 r - 2 cosh 2 r + 1) 

Z Z Z 

— 021(2 — — ) (1 — — )(coshr — /3)~^ sinh 2 r + n(coshr — /3) 1- ^ coshr 

Z L __■ 

+ a 20 (coshr - /3) 2 "^ = 6 2 (1 - /3 2 ) 2 (coshr - /J) - '" 1 , 
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i.e., 



62(1 - /3 2 ) 2 = (2 - \ ) [-2(n + 1) (l - |) (coshr - 0) s 



n. 



+ (1 - -){n l + 4n + 2)(coshr - (3) 1 coshr 
+n 2 (coshr - /3) 3 coshr + 2(n+ 2)(1 - ^)(-^) (coshr - /3)(cosh 3 r - cosh 
+(l-f)(-f)(-f-l)(l-cosh 2 r) 2 

— a2i(2 — ^) (1 — ^)(coshr — /3) 2 (cosh 2 r — 1) + n(coshr — /3) 3 coshr] 
+ a 2 o(coshr - (3) 4 . 



It follows that 



6 2 (1 - /3 2 ) 2 =a 20 (/3 4 - 4/3 3 cosh r + 6/3 2 cosh 2 r - 4/3 cosh 3 r + cosh 4 r) 



+ (2 ) (na 2 i — n 2 ) (/3 3 cosh r — 3/3 2 cosh 2 r + 3/3 cosh 3 r — cosh 4 r) 

+ (2 - ^)(1 - ^) [a 2 i - 2(n + 1)] (/3 2 - 2/3 cosh r + cosh 2 r) 

+ (2 - -)(1 - -) [(n 2 + 4n + 2) - a 2 J (/3 2 cosh 2 r - 2/3 cosh 3 r + cosh 4 



+ (2-£)(l-!f)(~)(n + 2) 



- + 2/3 coshr 
2 



cosh r — 2/3 cosh r + - cosh r 



Thus we have the following system of equations with unknown constants a 2 i, a 2 o and 6 2 : 

-6 2 (1 - /3 2 ) 2 + /3 4 a 20 + (2 - f )(1 - f )[a 21 - 2(n + l)]/3 2 

-i(2-f)(l-f)(-f)(n + 2) = 0, 
-4/3 3 a 20 + /3 3 (2 - § ) [na 21 - n 2 ] - 2/3(2 - f ) (1 - f ) [a 21 - 2(n + 1)] 

+2/3(2-f)(l-f)(-f)(n + 2) = 0, 
6/3 2 a 20 - 3/3 2 (2 - f )(na 21 - n 2 ) + (2 - f )(1 - f ) [a 21 - 2(n + 1)] 
+/3 2 (2 - f )(1 - f ) [(n 2 + 4n + 2) - a 21 ] 
-(2-f)(l-f)(-f)(n + 2) = 0, 
-4/3a 20 + 3/3(2 - § )(na 21 - n 2 ) - 2/3(2 - § )(1 - f ) [(n 2 + An + 2) - a 21 ] 

-2/3(2-f)(l-f)(-f)(n + 2) = 0, 
a 20 - (2 - § )(na 2 i - n 2 ) + (2 - § )(1 - f ) (n 2 + 4n + 2) - a 2i ] 
+ 3(2-S)(l-5)(-5)(n + 2) = 0, 
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n 2 -2n-4 n(n - 4)(n 2 - 2 2 ) 
a>2i — 2 ' a 20 - 02 24 



(hi) For fc = 3 and ip 3 (r) = (1 -/3 2 )( n " 6 )/ 4 [coshr - f3] 3 5 ? we have that 



A r ^ 3 = -(l-/3 2 ) (n " 6)/4 (3--) (2- -)( coshr 2 sinh 2 r + n( coshr - /3) 2 cosh-, 



A^ 3 =(1 - /*>)<»-«>/< (3 - [(2 - p (1 - p ( - p ( coshr - 0) ' 
+2(n + 2) (2 - -)(1 - -) (coshr - 0)~* (sinhr) 2 coshr 
+n(n + 2)(2- -)( coshr - /3) 1_ ^ cosh 2 r 



Us 



2 sinh r 



2' 



+2(n + 1) (2 — — ) ( cosh r — 0) 2 sinh 2 r + n 2 ( cosh r — 0) 



2 21. 

2 coshr 



and 



=(1 _ ^)-? (3 _ ») [ (2 _ ») (2 _ i) (») (» + i) ( » + 2) ( cosh r - /J) 



sinh r 



-3(n + 4)(2-^)(l-^)(^)(^ + l)(coshr-/3)' 



'(sinhr) 4 coshr 

\ 2 2 



-3(n + 2)(n + 4)(2- ^)(l- ^)(- ^)( coshr-/?) 2 (sinhr) 2 cosh 2 r 

— ■ — ■ — ■ 

+ (6n+14)(2- -)(1 --)(-) (coshr -/J)"*" 1 sinh 4 r 
-(6ra + 14)(ra + 2)(2- ^)(l- ^) (cosh r - (sinhr) 2 coshr 



-71 



Tl Tl 

(n + 4)(n + 2)(2--)(l--)(cosh, 



/?) 2 cosh 3 r 



-(n + 2)(3n + 2)(2- -)(l- -) (coshr - 0)'* (sinhr) 2 coshr 



-n(n + 2) (3n + 2) (2 - -) (cosh r - 0) 1_ * cosh 2 r 

2 



-4(n + 1) 2 (2 - ^) (coshr - /3) 1 2 sinh 2 r - n 3 (coshr - 0f 2 coshr 



By substituting these results into 



Tl + 6 

n — Q 



A r if; 3 + a 32 A r j/) 3 + a sl A r ip 3 + a 30 ip = b 3 ip 3 , 
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we have 

(1 - f3 2 ) 3 b 3 = a 30 ( cosh r - 0) - (3 ) [na 31 - n 2 a 32 + n 3 ] ( cosh r - /3) 5 cosh r 

- (3-|) (2-^) [a 31 -2(n + l)a 32 + 4(n+l) 2 ] ( coshr - /3) 4 (cosh 2 r - 1) 

+ 2) (3 - ^) (2 - [a 32 - (3n + 2)] ( coshr - 0) 4 cosh 2 r 
+ {n + 2) (3 - |) (2 - |) (1 - |) [2a 32 - (3n + 2)] (coshr - /3) 3 (cosh 3 r - coshr) 
+ (3 - \ ) (2 - \) (1 - ( - \ ) [a 32 - 2(3n + 7)] (coshr - /3) 2 (cosh 2 r - I) 2 

- (n + 2) (3 - \) (2 - \) (1 - 2) [_»(» + 2) (cosh 2 r - /?) 3 

+ -n(n + 4) (cosh 2 r — /3coshr)(l — cosh 2 r) 2 
+ -n(n + 4)(coshr — /3) 2 (cosh 2 r — cosh 4 r) 

+2(3n + 7) (coshr - /3) 3 (cosh 3 r - coshr) + n(n + 4) (coshr - /3) 3 cosh 3 r] . 

Letting the coefficients of 1, coshr, ••• , cosh 6 r are respectively zero, and then solving the 
corresponding system of equations, we obtain the unique solution: 

3n 2 - 6n - 32 3n 4 - 12n 3 - 52n 2 + 128n + 192 

^4-32 = : , «31 = 



030 = h = 



4 16 

n(n-6)(n 2 -4 2 )(n 2 -2 2 ) 
2 6 



(iv) By repeating the same process as in (i), (ii) and (iii), we can get all the constants b k 
and afc m , (k = 1, 2, • • • ; < m < k — 1). For example, we get that 

3 3 27 

a 43 = -(n 2 -In- 20), a 42 = -n 4 - -n 3 n 2 + 30n + 108, 

8 2 2 

n 6 3 29 

a 4 i = + -n 5 + 3n 4 n 3 - 39n 2 + 108n + 144, 

16 8 2 

n(n - 8) (n 2 - 6 2 ) (n 2 - 4 2 ) (n 2 - 2 2 ) 
a 4 o — 64 — 



and 



054 = 



— aso — B 5 



(-l) k a k0 = b k = 



2 8 

-5n 2 + lOn + 160 



4 

_ n(n - 10) (n 2 - 8 2 )(n 2 - 6 2 )(n 2 - 4 2 )(n 2 - 2 2 ) 

n[n - 2k] [n 2 - (2(fc - l)) 2 ][n 2 - (2(fc - 2)) 2 ] ■ ■ ■ [n 2 - 2 2 ] 

2^ 
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for all k = 1,2, □ 

Recall that for the ball model of the hyperbolic space HP, its standard metric is 

where 6 is the Euclidean metric of IP. Now, we introduce the following operator on (HP, h) 
by 

k-i 

(2.12) P k := A h h + ]Ta fcm A£\ 

where the constants a,k,k-i, • • • > a fco are given by Lemma 2.2, and Ah is the Laplacian on HP 
with the metric h (which has the local representation: 



A u 1_ d f /fo h ij du \ 

v'77 <>■!■; V 



for every it G C°°(H n )). We shall call Pk the fe-th standard operator on the hyperbolic space 
(H n ,/i), and ak m (0 < m < k — 1) the coefficients of Pk- It is easy to see that Pi is just 



the Yamabe operator — n( - n ~ 2 ^ Q n H n (see [Ai], [A2], [LP] and [Cha]), and P2 simply is 
the Paneitz-Branson operator on H n (see [DHL] or [Cha]). The general Yamabe and Paneitz- 
Branson operators are defined as follows (cf.[Pa], [DHL] and [Cha]): Given (M,g) a smooth 
n-dimensional Riemannian manifold, and let S g be the scalar curvature of g, and let Ric g be 
the Ricci curvature of g. Then the Yamabe operator L g , the second order operator whose 
expression is given by 

n-2 
4(n- 1)' 

The fourth order Paneitz-Branson operator Z g is given by 



Yg = AgU + — -SgU. 



ZgU = A g u - divg [(r n S g g + r] n Ric g )du] H —Q g u, 



where 



and 



) 1 A q 1 - 4n 2 + 16n - 16 c2 2 2 

' 9 ~2(n-l) 9 * 9+ 8(n-l) 2 (n-2) 2 ^ (n - 2)^°^ 



_ (n-2)^+4 
' n ~ 2(n-l)(n-2) ' 



^ n — n-2 • 

Note that the /c-th standard operator P^ on (H n , /i) is a 2/c-order linear differential operator 
defined on H n . Surprisingly, we have the following recursive formula, which shows that the 
k-th standard operator Pk on HP can be calculated by the first standard operator Pi: 



14 
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Theorem 2.3. Assume that (H n , h) is the hyperbolic space. Let P 1 = A h - be the first 

standard operator on H n , and let P& be the k-th standard operator on HP whose coefficients is 
given by Lemma 2.2. Then 



(2.13) 



P fc = P fc _i [Pi + fc(fc - 1)] , fc = 2,3, 



and therefore 



(2.14) 



P fc = P x [Pi + 2] [P x + 6] • • • [Pi + k(k - 1)] 



/or aZZ k = 1, 2, • • • . 

Proof, (i) For = 2, we have 



Pi (Pi + 2) 



A, 



n(n-2) 



Ak _ ( n(n-2) _ ., 



a2 /2n(n-2)-8\. n(n - 2)(n 2 - 2n - 8) 

= A/l ~l 4 J + 16 

a2 fn 2 -2n-4\ A n(n - 4)(n 2 - 2 2 ) 



(ii) For k = 3, it is easy to verify that 



P 2 [Pi + 6] 

\ 2 n 2 -2n-A n(n - 4)(n 2 - 2 2 ) 
A /i o A /i H 



- A 3 



2 4 

n(n - 2) - 24 n 2 - 2n -4 



A 



+ 



(n 2 - 2n - 4)(n 2 - 2n - 24) + n(n - 4)(n 2 - 2 2 ) 



8 



16 



A, 



n(n - A)(n 2 - 2 2 )(n 2 - 2n - 24) 
64 

. o 3n 2 - 6n - 32 A , 3n 4 - 12n 3 - 52n 2 + 128n + 192 A 
A 3 , - A A 2 + A 



16 



h 



n(n-6)(n 2 -4 2 )(n 2 -2 2 ) 
2 6 



Ps 
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(iii) For k = 4, we have 



P 3 [Pi + 12] 

3n 2 - 6n - 32 



A 3 - 



A 2 3n 4 - 12n 3 - 52n 2 + 128n + 192 A 
A£ + — A ft 



n(n-6)(n 2 -4 2 )(n 2 -2 2 ) 



26 



x 



A 



9 



16 

n(n — 2) 



12 



-A 4 



n 2 - 2n - 48 3n 2 - 6n - 32 
: H : 



A 3 , 



+ 



(3n 2 - 6n - 32) (n 2 - 2n - 48) + 3n 4 - 12n 3 - 52n 2 + 128n + 192 



A 2 , 



+ 



16 16 

(3n 4 - 12n 3 - 52n 2 + 128n + 192)(n 2 - 2n - 48) + n(n - 6)(n 2 - 4 2 )(n 2 - 2 2 ) 

26 

n(n - 6)(n 2 - 4 2 )(n 2 - 2 2 )(n 2 -In- 48) 
2« 



A, 



(3 3 27 \ 

-n 4 - -n 3 - —n 2 + 30n + 108 j A 2 h 



+ 
--Pa- 



n b - 3n 4 + — n 6 + 39n z - 108n - 144 A h 

16 8 2 

n(n - 8)(n 2 - 6 2 )(n 2 - 4 2 )(n 2 - 2 2 ) 
2 8 



(iv) By checking the results that are obtained from Lemma 2.2 for all k, we get the desired 
conclusion. □ 

Remark 2.4. Now we can easily calculate the coefficients ak m of the k-th standard operator 
Pk on H n by the recursive formula (2.14) instead of that complicated method as in Lemma 2.2. 



The following theorem give a basic transformation law for P k : 

Theorem 2.5. Let (HP,/?,) be the hyperbolic n-space, and let Pk be defined as above. Sup- 
pose a : M n — > H n is the conformal transformation defined by (2.1). Assume that ^fc(x) = 



l-\x\ 2 ) 



(n-2fe)/2 



Then for any u e V{W), 



(2.15) 



n + 2k 



4"" 2fe \ p k{u o a)] = A k (£ k ■ (u o a)), for iGB n , 



where V(M n ) denotes the space of smooth functions with compact support in EP , and A k is 
the standard k th -iterated Laplacian in R n as in Introduction (see, §1). 
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Proof. We shall prove this theorem by principle of mathematical induction. For convenience, 
we shall simply write Pk(u o a) as PkU in B n . For k = 1, noting that 



Vh 



\-\x\ 1 



we have 

A h u 



1 du d ( 



E 

m'=i 

n 

E 

n 

E 



dxidxj \fhdxjdxi 
1 — b| 2 x 2 d 2 u 



1 — IxPxn <9l(. 8 



1 — la; 



2 ^ 2 d 2 u 



dxidxj 



Sij + (n - 2) 



dxj dxi 



1 — Ixl 



9-u 



f 1 - 


a; 




A« + (n-2) 






V 2 J 


V 2 



n n 

EOU 

i=i 1 



so that 



1 - Ixl 



n + 2 
2 



n(n - 2) 
A^u u 



1 - \x\ 
2 



ra + 2 
2 



1 — \x\ 



■) Au + (n-2)( 



1 - Ixl 



: )E 



<9ii n(n — 2) 
<9x 3 - 4 



u 



du n(n-2), 2 , s±i 



v i — |x| 2/ v l — |xr y t-f 4 x i-\x\ 2 ' 



i=l 



On the other hand, 



(s-0 i 



•^ij ^ — 1) 2, • • • , n 



and 



so that 



dx 2 



n --l 
2 



1 - x 



n 



2/\l-i 



Tl + 2 

2 



.17 



A6 = -E 



c> 2 6 _ n(n-2) , 2 ^ 
4 4 -Id 2 ' ' 



, 9x 2 



SHARP fc-ORDER SOBOLEV INEQUALITIES IN H n 



17 



It follows that 



n rj2/ t \ 



i=i 



i=l 



n(n-2) ( 2 



it. 



Thus 



Tl + 2 

er 2 (A«) = A«o 



i.e., (2.15) holds for k = 1. We now suppose that for > 1, the law (2.15) holds, that is, 



(2.16) 



Tl + 2fc 

2 \ 2 



fe 



2 N 2 " 



1 - \x\ 2 



u 



1- M 2 . 

Our purpose is to prove that (2.15) still holds for k + 1. By (2.15) and (2.16), we have that 

x+2(fc+l) / 2 



^ + 2(fc + l) 
2 



n+2(fc + l) 

2 x 



1 - \x\ 2 
2 



1 - |x| 
[P fc (Pi + fc(fc + l))u 



(Pfc+iw) 



1 — I 1 2 y V 1 — la; 
2 



1 — |x| 
2 

1 — |x| 
2 



A 



Tl + 2fc 

2 ; 2 [Pfc(Pl + fe(fe+l))«] 

2 



A fc 



1 — \x\ 
2 

1 — \x\ 



(Pi + fc(fc + 1)) 



Ti — 2fc -i 



(P lU )+k(k + l)( — 



X 



ix 



9 n-2fe 



9 n-2 



u 



On the other hand, we can verify, by a direct calculation, that for any positive integer k > 1, 

A k+i 



9 Tl-2(fc + l) 

.. — 2 



?1 



-fc-i 



+^+l)( T3T ^)- 



x\ 

n — 2k 



9 n-2 



\x\ 



u 
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Hence 

Tl + 2(fc + l) 

which completes the proof. □ 

Remark 2.6. i) The identity (2.15) can also be written as 

_ n + 2k n—2k 

(P k u)oa = J a 2 " A k [j a 2n uoa], forueV(M n ), 

where J a is the Jacobian of a. 

(ii) The k-th standard operator P k on the hyperbolic space (H n , h) is essentially the power 
A k of the positive HP Laplacian lifted to the hyperbolic space via the conformal transformation 
a, using the identity (2.15). We can also call P k the k-th intertwining operator on (HP, h). 



§3. Sharp /c-order Sobolev inequality in the hyperbolic space HP 

Lemma 3.1. Let (HP, h) be the hyperbolic space, n > 2k. Suppose that there exist real 
constants b' k and {a' krn }o<m<k-i such that for any u G V(M n ), 

(3.1) b' k ([ \u\«dV h ) /q <[ \A k h /2 u\ 2 dV h +J2*'km[ \A™ /2 u\ 2 dV h , 



where q = (2n)/(n — 2k). Then b' k < where A k is the best k-order Sobolev constant in M. n . 

Proof. This proof follows from the lines of [Ai] (also see [Hi]). Suppose by contradiction 
that there exist b' k and {a' km } satisfying b' k > such that inequality (3.1) holds for any 
u e V(M. n ). Let y G H n . It is easy to see that for any e > there exists a chart (U, 4>) of EP at 
y, and there exists r > such that <p{U) = B T (0) (here B T (0) is the Euclidean ball of center 
and radius r in R n ), and such that the components hij of h in this chart satisfy 



(l-e)S ij < < (l + e)A 



as bilinear forms. Choosing e small enough we can get by (3.1) that there exist To > 0, b k and 
{a km } satisfying b kk > such that for any r G (0, r ) and any u G V(B T (0)), 



b' k ' ( [ \u\ q dx) /9 < [ \A k h /2 u\ 2 dV h + V a' k ' m [ \A m / 2 u\ 2 dx. 

Applying Nirenberg's lemma (see [N] or [ADN, Lemma 14.1]), we find that there exists a 
constant c depending only on r such that for any < m < k — 1, 

/ \A m / 2 u\ 2 <r[ \A k / 2 u\ 2 dx + c [ \u\ 2 dx. 

</R™ JR n JR n 
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Noting that 1/q = 1/2 — k/n, it follows from Holder's inequality that 

2/<z 
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/ \u\ 2 dx < |P T (0)| 2fc/n ( / \u\ 
Jb t (o) \Jb t (o) 



where |P T (0)| denotes the volume of the ball B T (0) in M. n . Therefore by choosing r small 
enough, we obtain that there exist r > and b' k n > such that for any u G V(B T (0)), 



2/q 

u\ q dx) < I \A k / 2 u\ 2 dx. 



For any u E V(R n ), let us set u v (x) = u(rjx), r\ > 0. Take r\ large enough, u v G V(B T (0)). 
Thus, 

2/q 



\u v \ q dx) < / \A k / 2 u v \ 2 dx. 



But 



while 



\u v \ g dx 



2/q 



(2n ^ q / \u\«dx 



2/q 



[ \A k / 2 U71 \ 2 dx = V 2k - n [ \A k / 2 u\ 2 dx. 
In view of 1/q = 1/2 — k/n, we have that for any u G V(R n ), 



1/9 

u\ q dx) < I \A k ' 2 u\ 2 dx. 



Since b'{/ > such an inequality is contradiction with (1.5). This completes proof of the 



lemma. □ 



We have the following sharp /c-order Sobolev inequality on H n : 

Theorem 3.2. Let (H n , h) be the hyperbolic n-space, n > 2k, and let q = (2n)/(n — 2k). 
Assume that Pt is the k-th standard operator on M n , that is, Pk = Pi (Pi + 2) ■ • • (Pi + k(k — l)) 
with Pi = A h - n(n ~ 2) . Then, for any u G V{W l ), 



(3.2) 

or equivalently, 
(3.3) 



2/q 

\u\ q dV h ) < A fc / (P k u)udV h , 



2/q 

\u\ q dV h J < A k 



I (K /2 u\ 2 + k ^a km \A™/\\ 2 )dV h 
J ^ n V m=0 / 
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where is the best k-order Sobolev constant in M. n (see, §1) and Ofc m are the coefficients of 
the standard operator P k . Moreover, for any < j3 < 1, if 



(l-/3 2 )^ 



up(r) = n _ 2k , r e [0, +oo), 

(cosh r — p) 2 

where r is the distance from on HP , then 

(34) Um f nn (P k up)updV h = . nf / Hn (P fc u)udVfc 



and 

(3.5) P fc (^^NM) = ^ K 1/9 ^(r)) rl , < r < + 



oo. 



Proof. Recall that 



dVh = r-P7 dx. 



n 



1 - \x\ 

where dx is the volume element of the Euclidean space lR n . It is clear that for every u G P(H n ), 

j mn (p k u)udv h __ /if. (i A t /2w i 2 +Etrio a fc-i A r /2w i 2 )^ 

and 

/ M „(P fc u)udVfc / Bn (P fc «)«dVh 



(3.7) 



where it and P k u in B n mean wo a and P k (uoa) respectively, and a : B n — > H n is the conformal 
mapping defined by (2.1). By Theorem 2.5 we have 

n + 2k 

(3.8) Q- 2k {P k u) = A k (Cku) inl", 

n — 2fc 

where = ^ !_ 2 x p ) 2 > and A fc is the /c th -iterated standard Laplacian in R n as in §1. 

Multiplying ^ k u to equation (3.8) and then integrating the result in B n (here B n is the n- 
dimensional unit ball in M n ), we get 

(3.9) / ( 2 ) {P k u)udx = [ (Cku){A k (Cku))dx. 
Jn n V 1 ~~ \ x \ J JM n 
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Note that 



(3.10) / (t^T-u) (P k u)udx= [ (P k u)udV h . 

JB n \ 1 — \ x \ J JM n 

By applying integration by parts to the right-hand side of (3.9), we have 

(3.11) I (Z k u)(A k (Z k u))dx= j \A k / 2 {t k u)\ 2 dx. 

It follows from (3.7)— (3.11) that 

f mn (P k u)udV h _ J B JA k / 2 ^ k u)\ 2 dx 



(3.12) 



{f mn \u\idV h ) 2/q (/ B je fc u|«da:) 2/ ' 7 
From Lemma 3.1, we see that 

Jiffi" (\ A h 2u \ 2 + Et= 1 a krn\A™ /2 u\ 2 ^dV h 1 

uevl^Mo} (J H „ \u\idV h f q " A 

Suppose by contradiction that 

kAK /2 n\ 2 +EtU k rn\Af\\AdV h 1 
mf 77 " < U-> 

uev(M~)\{o} (j^ \u\idV g ) 2/q A fc 

and let u E £>(H n ), u ^ satisfy 

, 3 13 s fm \K /2u o\ 2dv h + Efc=j a k m\A™ /2 u \ 2 dV h < ^ 

By (3.6), (3.12) and (3.13), we have 

/ B JA*/ 2 (&ti )| 2 da: ^ 1 

(JbJ^o|^) 2/9 Afc ' 

Clearly, £ k u G P(B n ). But 

/BjA*/ 2 (&uo)| 2 <fa > . nf J, B \A k ' 2 w\ 2 dx > j_ 
(/ B |^ |^) 2/9 " <"«>&.)\{o} (J B | w |^x) 2/q ~ A fc ' 



fc 
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This is a contradiction, which shows 



inf 

uET>(U n )\{0} 



Im n (l 



A 



fc/2 



U\ 



. Y^fc— 1 



A 



m/2 



?1 



2/9 



1 



Hence we get (3.3). 

Finally, by the conformal map a from (M n ,8) to (H n ,/i), the function Gp(x) (see (2.3)) 
defined in B n is lifted to up{r) (i.e., ip k (r)), and (3.4) is true. (2.10) of Lemma 2.2 also shows 
that equation (3.5) holds. □ 

It is easily seen from Theorem 3.2 that for any constants r krn , m = 0, 1, • • • , k satisfying 
r kk > Afc and r krn > A k a km (m = 0, 1, • • • , k - 1), we have 



(f \ u \w h ) 2/q < [ (j2^\ A h /2u \ 2 ) dV ^ 

\Ju n J Ju n V m=0 / 



for any u G T>{W i ). Lemma 3.1 implies that in order to make the above inequality hold, it 
is necessary that r kk > A k , i.e., A k can't be lowered. Combining this fact and the following 
theorem, we shall conclude that A k , A k a k ,k-i, • • • ,A k a k o are the best constants since they 
can't be lowered. 

Theorem 3.3. Let (H n , h) be the hyperbolic space, n > 4k — 2, and let q = (2n)/(n — 2k). 
Assume that Ti (i = 0, 1, • • • , k — 1) is the operator defined by 



k i 

%= a km^h + T ^ A "' i = 0,l,--- ,k~l, 

m=i+l m=0 



with a kk = 1- For each fixed i G {0, 1, • • • , k — 1}, there exist real numbers r^o, • • • ,T k i such 
that for all u G V(U n ), 



MlL(H») / (Tiu)udV h 



(3.14) 



=A fc 



y~] a km \A™ /2 u\ 2 dx + y~] T km I \A™ /2 u\ 2 dx 

«=i+l J ^ n m=0 



if and only if r k i > a k i, where {a km } are the coefficients of the k-th standard operator P k , and 
A k is the best Sobolev constant in M n . 

Proof. For any fixed i G {0, 1 • • • , k — 1}, if r k i > a k i, the result immediately follows from (3.3) 
of Theorem 3.2 because we may take T km = a km , m = 0, 1, • • • , % — 1. 
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Suppose, on the contrary, that there exist r^o, • • • , Tfei satisfying < a^i such that (3.14) 
holds for all u E V(W l ). For < f3 < 1 real, and r the distance to the origin on HP, we let 
up(r) be as in Theorem 3.2, that is 



(cosh r — (3) I!_ 2 2iL 



By (2.8), up{r) is lifted by the function 



G p (x) = 



l-0\ 1/2 (l + m 



.7" 



1+P 



) \ 



k— — 



and 



I \up{r)\idV h = I \Gp{x)\*dx= I (\ 
Ju n Jn n Jn n V ' y 

i-PY n/2 f 1+1 T-P 



n\ -n/2 /i I l+ff |„|2\ -„ 



-<^n-l 



o V1 + /3 



^s 2 



„ / 1 +' 9 

/ (1 + ^ 2 )" 
./o 



Hence 



(3.15) 



lim / \up(r)\ q dV h = lim 2 n Wn _x 

Z 3 ^ 1 Ji» P^ 1 JO 

=Tuj n - X \ (l + z 2 )- n z n - 1 dz = 2 n - 1 u n - 1 
Jo 



r(n/2)r(n/2) 



r(n) 



UJr, 



which also implies that J m „ \up{r)\ q dVh is increasing with respect to /?, (0 < /3 < 1). According 
to (2.10), we have 

fe-i 

A* Uj9 (r) + }^ a fcm AP W/3 (r) = ^(r)- 2 *, 

m=0 

i.e., 

PkUp{r) = bkujf 1 , < r < +oo, 
where Pfc is the k-th standard operator on HP. Thus 

It follows that for any f3 G (0, 1), 



< r < +oo. 



\un 1,q up{r)\\\ 



L<?(H") 



(^(c-^Xr)),) 2 ^ 



1 



k \ JU 



{u-^up{r)) q dV h 



1-2/9 



1 



A, 1 n 



\ 1-2/9 i 

l«/j(r)|W h J <x -. 
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Since 

i 

TiUp(r) = P k up(r) + ^ ( T k™ ~ a km ) A%(up(r)), 
we obtain that for any < f3 < 1, 

J H „ (r^- 1 ^^))^- 1 /^^^)^ _ / Hw (P^n 1/ %(r))(a;n 1/q tx /3 (r))dV r fe 
\\un 1/q Uf3{r)\\ 2 Lq{m \\^n 1/q u p {r)\\ 2 Lq{Wl) 

/h« (eL=o( t ^ - afcm)A^(wn 1/ %(r))^(wn 1/<? 'U / 3(r))rfV^ 
||wn 1/g w /3 (r)||| 9(H „ ) 

(3.16) 

! /h- (ll=ofam -a fan )A™(^ 1 \ /3 (r))j(a; n - 1/ V(r))^ 
Clearly, if i = 0, then (3.16) implies 

llwn^VMHLoin") Afc 
This contradicts (3.14) because we can choose a function vp G £>(IHI n ) (see below) such that 

jgn (Tjvp)vp dV h J_ 

This proves the conclusion of the theorem. 

Now we consider the case i > 1. It is obvious that 

^e^gr) m = ,i,..., i -i. 

Therefore, for /5 sufficiently close to 1, by (3.15) 

/h« ( ELo( r tm - a fcm ) A^(^ 1/9 W/3 (r))^ (w^\(r))(^ 

||Wn 1/g W/3( r )llL(H») 



m=0 
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where 



lim 



^ EL= ( T ^ - a km) / H „ |A^ /2 (w n 1/9 W/3( r ))| 2 ^ 



0. 



For each m G {0, 1, • • • , i — 1}, it follows from Nirenberg's lemma (see [N] or [ADN, Lemma 
14.1]) that for any g > 0, there exists a constant c m depending only on g, m and i such that 

(3.17) f \A™ /2 u\ 2 dV h < g f \A l l 2 u\ 2 dV h + c m [ \u\ 2 dV h 

Jw n Ju n Ju n 

for all u G W k ' 2 (M. n ). We can choose g > small enough such that 

i-1 

Tki - dki + Q ^2 \ Tkm ~~ akm \ < ®- 

Thus 

f Mn (T^K^r)) {un 1/q u p {r))dV h 



m=l 



u n 1/q up(r)\\ 2 Lq{w 



< T- + 



i-i 



Afk- 1/ %W)| 2 ^ + /2(/3) 



(3.18) 



where 



+4 



K 1/q M r )\ 2dV h 



lim 

0-1- 



/ 2 (/5) 



- a kl + gY? m =i \ T km ~ a km \ ) / e „ | A 1 / 2 (w n (r) ) 1 2 cft4 



o, 



and is a constant depending only on g and z. 
Recall that under the confer mal map a, we have 



cosh r — 1 = 



2 


x 


2 


1 - 




£ 


2 



, r G [0, +oo), x G B n . 



Since 



Up (r) =(1- [3 2 )—[coshr - [3] 2 

= (1 - /3 2 )^ [(1 -0) + (coshr - 1)]' 
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we find that the function up(r) can be written as: 



up{r) = (1 - /3 2 ) 

It follows that 

|2jt^ / /1 o2 



n — 2k 

2\ —3— 



(1-/3) + 



2 


x 


2 


1 - 




X 


2 



x e 



\u p {r)\'dV h = 



(1-/3) + 



2 


x 


2 


1 - 




X 


2 



2fc-n 



1 - X 



dx 



Wn-l(l 



Jo 



(1-/3) + 



2s 2 n2fc " n 



1 -s 2 



1-s 2 



= 2^ n _ l( l - /3 2 )^ jf 1 [(1 - /?)(! - * 2 ) + 2. 2 ] 2 ^ 



2fc 



Making the change of variables t = a/(1 + /3)/(l — ~0) s, we have that 
/ \up{r)\ 2 dV h 

1+/3 



(! + *»)' 



Since 



(1-/3) + 



2 


x 


2 


1 - 




X 


2 



t— 2. — 1 

2 4x, 



(1- |x| 2 ) 2 ' 



it follows that 



= (id£L ) «. (1 _ /P) -=i- (*_»)> 



(i - IS) + 



2 


X 


2 


1 - 






2 



2A:-n-2 



(r 



- xr 



X 



so that 



/ \V h u p {r)\ 2 dV h 

=(1-^)^(^-1)7 

t/ IBL 



(1-/3) + 



2 


X 


2 


1 - 




X 


2 



n 2fc-n-2 



.7" 



2"(l-/3 2 )^(fc-l)^„- 1 



X 



[(1 - /3)(1 - s 2 ) + 2s 2 ] 2k ~ n - 2 (-^) 2k - 2 s n+1 ds. 
Jo 1 — s 
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Again, by the substitution 

t= + s, 

we have 

/ |V^(r)| 2 ^=2-(/ C -^) 2 a; n _ 1 (l-/? 2 )- fe - 1 (l-/?) 2fe 



(1 H- t 2)"- 2fc + 2 ' 



1+/3 

It follows from the dominated convergence theorem, if n > Ak — 2, 

A^-io U-J^fvvi-Sftv (i+t 2 )- 2fc " ' 

while 

-vW / 1 \ 2fc - 2 t n+1 dt r + °° t n+1 dt 



lim 



The latter integral on the right-hand side is a finite positive constant for n > Ak — 2. On the 
other hand, since 

lim 2^ n _ 1 (l-/3 2 )- fc " 1 (/3-l) 2fc _ 1 



0-1- 2»(jfe - f ) w n _i(l - /3 2 )- fc " 1 (l - P) 2k ( k ~ f ) 
it follows that 

(3.19) fin /-:'" /i(r r )i ;^ = 0, 
which implies 

(3.20) lim l ^ 1/ > (r)|2#fc = 0. 

/?-i- / H „ iv^- 1 /^))! W h 

By taking m = 1 and replacing u by it/j(r) in (3.17), we have 



n\2 ' 



J^IA^^-^Xr))! 2 ^ / Hw I^^XOIW, 

1 — Q T~i rCi -7 . 

S nn \V h {un 1/q up{r))\HV h J Mn \V h (u;n 1/q up(rWdV h 



Letting j3 — > 1 , we find by this and (3.20) that 

(8.21) J H JAfKVV(r))|W t> ^ 

_ / H „|V, 1 K 1/ «^(r))|W ft - £> 
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Combining (3.18), (3.20) and (3.21), we obtain that if r ki < and n > 4k — 2, then for f3 
sufficiently close to 1, 

TH-a kl + g V \T km -a km \ ] / \A)! 2 (u;- 1 /«Uf } (r))\ 2 dV h + d' i [ \u;- 1/9 up(r)\ 2 dV h < 0. 

m=0 / " " 



From (3.18), it follows that for f3 < 1 sufficiently close to 1, 



'')lli<i(ii™)- 



Finally, for /5 sufficiently close to 1, since up(r) G W k,2 {M n ), we can choose a sequence of 
smooth functions Wj with compact support in H n such that 

lim \\up - Wj\\ W k,2 (mn ) = 0. 
Thus we can find an integer j such that 



/ {Ti(w jo )) {w jo )dV h < -T-\\w jo \\l q{m - 



Let us denote Wj by vp. This is in contradiction with (3.14), which proves the theorem when 
n>4k-2. □ 

Remark 3.4. i) For order k = 1, the inequality (3.3) had been given by Hebey in [Hi, HsJ. 

ii) As argument before, there is not extremal function for the sharp Sobolev inequality 
(3.2) (i.e., (3.3)) on H n . 
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